Towards augmenting dialogue strategy management with multimodal sub-symbolic context

Abstract. A synthetic agent requires the coordinated use of multiple sensory and effector modalities in order to achieve a social human-robot interaction (HRI). While systems in which such a concatenation of multiple modalities exist, the issue of information coordination across modalities to identify relevant context information remains problematic. A system-wide information formalism is typically used to address the issue, which requires a re-encoding of all information into the system ontology. We propose a general approach to this information coordination issue, focussing particularly on a potential application to a dialogue strategy learning and selection system embedded within a wider architecture for social HRI. Rather than making use of a common system ontology, we rather emphasise a sub-symbolic association-driven architecture which has the capacity to influence the ‘internal ’ processing of all individual system modalities, without requiring the explicit processing or interpretation of modality-specific information

RescueSpeech: A German Corpus for Speech Recognition in Search and Rescue Domain

Despite the recent advancements in speech recognition, there are still difficulties in accurately transcribing conversational and emotional speech in noisy and reverberant acoustic environments. This poses a particular challenge in the search and rescue (SAR) domain, where transcribing conversations among rescue team members is crucial to support real-time decision-making. The scarcity of speech data and associated background noise in SAR scenarios make it difficult to deploy robust speech recognition systems. To address this issue, we have created and made publicly available a German speech dataset called RescueSpeech. This dataset includes real speech recordings from simulated rescue exercises. Additionally, we have released competitive training recipes and pre-trained models. Our study highlights that the performance attained by state-of-the-art methods in this challenging scenario is still far from reaching an acceptable level

Towards long-term social child-robot interaction: using multi-activity switching to engage young users

Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI

The dynamic potential of topic and focus

Available from STL Prague, CZ / NTK - National Technical LibrarySIGLECZCzech Republi

Contextually Appropriate Ordering of Nominal Expressions

algorithm for word order Given: a list G of ordering constraints imposed by the grammar, a list L1 of constituents that need to be ordered, a list Delta giving ordering of CB constituents, create empty lists LC and LN % LC for CB items, LN for NB items repeat for each element E in L1 if E is CB, then add E into LC, else add E into LN. if the verb is CB, then Order the verb at the end of LC Order the remainder according to D else Order all elements in LC according to D % thus, if e precedes f in D, then e precedes f in LC except for the verb. if G is not empty then Order elements in L1 using ordering constraints in G The proposed ordering algorithm as such is the same for all the three languages under consideration. What may di#er is first of all the systemic ordering, and therefore the default ordering of non-bound elements. Further di#erences between the languages are encountered in the constraints on which elements can be ordered rather freely and which elements are subject to order..

Conditional Responses in Information-Seeking Dialogues

The paper deals with conditional responses of the form "Not if c/Yes if c" in reply to a question "?q" in the context of information-seeking dialogues. A conditional response is triggered if the obtainability of q depends on whether c holds: The response indicates a possible need to find alternative solutions, opening a negotiation in the dialogue. The paper discusses the conditions under which conditional responses are appropriate, and proposes a uniform approach to their generation and interpretation